Software-Defined Networking (SDN) is an evolving networking paradigm that still faces shortcomings that have to be overcome. A logical overlay network that is implemented over a physical network infrastructure of a hosting system for a tenant of the hosting system is a good example of SDN. A logical network, logically connects different virtual machines (VMs) of a tenant, which run on one or more host machines, to each other and to other end machines (i.e., physical and/or virtual machines) of other external networks (i.e., physical and/or logical networks). A logical network may logically connect different virtual servers, on which, a distributed application (e.g., a multi-layer web application) executes.
Conventionally, in order to balance the distributed application's load on the virtual servers that implement the different layers of the application, one or more load balancers distribute incoming service requests to the application (e.g., new network connection requests) between the different virtual servers. The conventional load balancers, however, distribute the new requests solely based on the network traffic routed to each virtual server since the load balancers lack visibility into the state of the VMs that host the distributed application. That is, in balancing the load of a distributed application, the load balancers disregard the current state of the VMs (i.e., virtual servers) and the load on each VM and across the VMs of a host machine. For example, a load balancer may send a new network connection request to a VM only because the network traffic forwarded to the VM is light, while CPU usage of the VM is very high, or available virtual memory of the VM is very low.